Searching for Expertise: Experiments with the Voting Model

In an expert search task, the user’s need is to identify people who have relevant expertise to a topic of interest. An expert search system predicts and ranks the expertise of a set of candidate persons with respect to the user’s query. In this work, we propose a novel approach for estimating and ranking candidate expertise with respect to a query. We see the problem of ranking experts as a voting problem, which we model using adaptations of data fusion techniques. We extensively investigate the effectiveness of the voting approach and the associated data fusion techniques across a range of document weighting models, in the context of the TREC 2005 and TREC 2006 Enterprise track settings. The evaluation results show that the voting paradigm is very effective, without using any collection-specific heuristics. Additionally, we further analyse two main features of the voting model, namely the manner in which document votes are combined and the effect of the underlying document ranking. First, for the combination of document votes, we hypothesise that candidate with large profiles can introduce bias in the generated ranking of candidates. We propose and integrate into the model a candidate length normalisation technique that removes bias towards prolific candidate experts. Secondly, we investigate the relative effects of applying various retrieval enhancing techniques to improve the quality of the underlying document ranking, to investigate how each technique improves the retrieval effectiveness of the generated ranking of candidates. At each stage, we experiment extensively and draw conclusions. Our results show that the voting techniques proposed are indeed effective, across several different document weighting models and settings. Secondly, we see that candidate profile length normalisation can help improve retrieval accuracy when applied to the candidate profile sets. Lastly, we show that increasing the quality of the underlying ranking of candidates can enhance the retrieval accuracy of the generated ranking of candidates.